Image matching device, image matching method and image matching program

ABSTRACT

Image matching device  300  of the invention includes feature image extracting sections  303, 304  extracting one or more partial object images containing a local structural feature from an object image and extracting one or more partial reference images containing the local structural feature from each reference image, first image detecting section  306  setting each of the partial object images as an image of interest and detecting a first partial image most similar to the image of interest from a set of partial reference images, second image detecting section  307  detecting a second partial image most similar to the first partial image from a set of partial object images, and determination processing section  305  determining whether or not the image of interest matches the second partial image and outputting the result of the determination.

TECHNICAL FIELD

The present invention relates to an image matching technique.

BACKGROUND ART

Image matching is a technique that matches an image against one or morereference images to determine whether or not the image matches any ofthe reference images. Image matching technique of this type is used inmatching a facial image or a fingerprint image captured by an imagepickup device against registered images stored previously in a database,for example. In this example, the facial image and fingerprint image areimages to be matched and the registered images are reference images. Inconventional biometric image matching techniques, global structuralfeatures unique to a living individual (for example, the eyes, theeyebrows, and the mouth) are matched. Since global structural featuresare fixed in number and are in almost fixed positions, matching based onglobal structural features can be performed with relative ease.

However, it is difficult to perforin matching of images of livingindividuals that have extremely similar global structural features, suchas a twin, with a high degree of accuracy with the matching technique.Therefore, matching techniques based on acquired local structuralfeatures (for example, skin traits such as moles, freckles and wrinkles,and fingerprints), in addition to global structural features have beenproposed.

Related-art documents concerning the image matching technique includeJP2006-107288A (hereinafter referred to as Patent document 1),JP06-28461A (hereinafter referred to as Patent document 2) andJP2005-521975 (hereinafter referred to as Patent document 3).

Patent document 1 discloses a personal verification technique in whichskin texture such as moles, flecks and freckles are detected and thedetected pattern is matched against feature patterns registered in adatabase. Patent document 2 discloses a fingerprint matching techniquein which a window image is extracted on the basis of a global structuralfeature in a fingerprint image and matching is performed on featurepoints (minutiae) such as branches and ends of a fingerprint that appearin the window image. Patent document 3 discloses a personal verificationtechnique in which a reference image is searched, a group of pixels thatis the best match for a group of pixels in an object image (acquiredimage) is selected, and the probability that the relative locations ofthe selected group of pixels and the group of pixels in the object imagerandomly occur is determined.

DISCLOSURE OF THE INVENTION

However, local structural features are not always stable. The positionsand shapes of local structural features (for example, feature pointssuch as moles, flecks and freckles) can change due to external factorsand such changes can decrease the accuracy of matching. For example, ifthe expression of a subject changes from usual expression or theappearance of the subject changes from usual appearance due to shootingconditions when an image pickup device attempts to capture an image ofthe subject to acquire an object image, the accuracy of matching of theobject image will degrade.

To address the problem and improve the accuracy of matching, thetechnique disclosed in Patent document 3 selects a pixel group from areference image that matches a pixel group in an object image to searchthe reference image. The technique disclosed in Patent document 3 canprevent the above-mentioned decrease in matching accuracy caused byexternal factors. However, the processing load of the search is so largethat the speed of matching is disadvantageously decreases.

An exemplary object of the invention is to provide an image matchingdevice, image matching method, and image matching program capable ofimage matching processing based on a local structural feature with asmall amount of computation and a high degree of accuracy.

An image matching device according to an exemplary aspect of theinvention, which matches an object image against one or more referenceimages, includes: a feature image extracting section extracting one ormore partial object images containing a local structural feature fromthe object image and extracting one or more partial reference imagescontaining a local structural feature from each of the reference images;a first image detecting section setting each of the partial objectimages as an image of interest and detecting a first partial image mostsimilar to the image of interest from a set of the partial referenceimages; a second image detecting section detecting a second partialimage most similar to the first partial image from a set of the partialobject images; and a determination processing section determiningwhether or not the image of interest matches the second partial imageand outputting the result of the determination.

An image matching method according to an exemplary aspect of theinvention, which is for matching an object image against one or morereference images, includes: performing a feature image extracting stepof extracting one or more partial object images containing a localstructural feature from the object image and extracting one or morepartial reference images containing a local structural feature from eachof the reference images; performing a first image detecting step ofsetting each of the partial object images as an image of interest anddetecting a first partial image most similar to the image of interestfrom a set of the partial reference images; performing a second imagedetecting step of detecting a second partial image most similar to thefirst partial image from a set of the partial object images; andperforming a determination processing step of determining whether or notthe image of interest matches the second partial image and outputtingthe result of the determination.

An image matching program according to an exemplary aspect of theinvention, which causes a computer to execute a process for matching anobject image against one or more reference images, includes: a featureimage extracting step of extracting one or more partial object imagescontaining a local structural feature from the object image andextracting one or more partial reference images containing a localstructural feature from each of the reference images; a first imagedetecting step of setting each of the partial object images as an imageof interest and detecting a first partial image most similar to theimage of interest from a set of the partial reference images; a secondimage detecting step of detecting a second partial image most similar tothe first partial image from a set of the partial object images; and adetermination processing step of determining whether or not the image ofinterest matches the second partial image and outputting the result ofthe determination.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram schematically illustrating an imagematching system according to one exemplary embodiment of the presentinvention;

FIG. 2 is a flowchart schematically illustrating a process procedureperformed by an image matching device;

FIG. 3 is a flowchart illustrating a specific exemplary procedure ofdetermination processing;

FIG. 4 is a diagram illustrating a feature space for explaining imagematching;

FIG. 5 is a diagram illustrating a feature space for explaining imagematching; and

FIG. 6 is a flowchart schematically illustrating a process procedureperformed by an image matching device according to a variation of theexemplary embodiment.

DESCRIPTION OF SYMBOLS

-   100 measuring device-   101 image pickup section-   200 storage device-   201 image storage-   202 image correspondence table-   300 image matching device-   301 image-to-be-matched extracting section-   301A first image extracting section-   301B second image extracting section-   302 normalizing section-   303 feature quantity calculating section-   303A first feature quantity calculating section-   303B second feature quantity calculating section-   304 region extracting section-   304A first region extracting section-   304B second region extracting section-   305 determination processing section-   306 first image detecting section-   307 second image detecting section-   308 image matching section

BEST MODE FOR CARRYING OUT THE INVENTION

Exemplary embodiments of the present invention will be described belowwith reference to drawings. Like elements are given like referencenumerals throughout the drawings and repeated detailed description ofthe elements will be omitted as appropriate.

FIG. 1 is a functional block diagram schematically illustrating an imagematching system according to one exemplary embodiment of the presentinvention. The image matching system includes measuring device 100,storage device 200 and image matching device 300.

Measuring device 100 includes image pickup section 101. Image pickupsection 101 includes a solid-state image pickup device such as a CCD(Charge Coupled Device) image pickup device or a CMOS (ComplementaryMetal Oxide Semiconductor) image pickup device, a focus system whichfocuses incident light from a subject onto the solid-state image pickupdevice, and a signal processor which applies image processing to outputfrom the solid-stage image pickup device. Image pickup section 101 canoutput image data to image storage 201 or image matching device 300.Storage device 200 includes a recording medium such as a volatile ornonvolatile memory (for example a semiconductor memory or a magneticrecording medium) and a control circuit and a program for writing andreading data on the recording medium. Storage device 200 includes imagestorage 201 storing image data input from image pickup section 101 andimage correspondence table 202.

Image matching device 300 includes image-to-be matched extractingsection 301, feature quantity calculating section 303, region extractingsection 304, determination processing section 305, first image detectingsection 306, second image detecting section 307, and image matchingsection 308. All or some of functional blocks 301 and 303 to 308 may beimplemented by hardware such as a semiconductor integrated circuit or bya program or a program code recorded on a recording medium such as anonvolatile memory or an optical disc. Such program or program codecauses a computer including a processor such as a CPU (CentralProcessing unit) to execute image matching processing of functionalblocks 301 and 303 to 308.

A configuration and operation of image matching device 300 will bedescribed below with reference to FIGS. 2 and 3. FIG. 2 is a flowchartschematically illustrating a process procedure performed by imagematching device 300. FIG. 3 is a flowchart schematically illustrating aprocedure of determination processing (step S108 of FIG. 2) performed bydetermination processing section 305.

Image-to-be-matched extracting section 301 includes first imageextracting section 301A, second image extracting section 301B, andnormalizing section 302. First image extracting section 301A extracts afirst object region image from an input image captured by andtransferred from image pickup section 101 and provides the first objectregion image to normalizing section 302 (step S101). Second imageextracting section 301B extracts a second object region image from aninput image read and transferred from image storage 201 (the image is aregistered image registered previously in image storage 201) andprovides the second object region image to normalizing section 302 (stepS102). First image extracting section 301A and second image extractingsection 301B may extract the first and second object region images,respectively, on the basis of a global structural feature, a colorregion or the contour shape in the input images.

The input images transferred to image matching device 300 aretwo-dimensional images in each of which pixel values are arranged in atwo-dimensional array. The pixel values are not limited to any specificvalue but may be any value in any color space. For example, the pixelvalue may be a value in an RGB color space or may be a luminance value(Y) or a color-difference value (Cb, Cr) in a YCbCr color space.

Normalizing section 302 performs at least one operation from among:position adjustment, rotation and scaling of a subject image in a firstobject region image on the bases of a global structural feature (forexample a feature such as the eyes, nose, and ears of a livingindividual) in the first object region image to normalize the firstobject region image to generate an object image SO (step S103). At thesame time, normalizing section 302 performs at least one operation fromamong: position adjustment, rotation and scaling of a subject image in asecond object region image on the basis of a global structural feature(for example a feature such as the eyes, nose, and ears of a livingindividual) in the second object region image to normalize the secondobject region image to generate a reference image SR (step S103). Whenan object to be examined is a facial image or a fingerprint image, thecenter of the eyes, eyebrows, nostrils, mouth, or facial contour, or awhorl of the fingerprint may be used as a global structural feature.When an object to be examined is an artifact, the shape of the artifactsuch as a cube or rectangle, or a feature of a logo may be used as aglobal structural feature.

Feature quantity calculating section 303 includes first feature quantitycalculating section 303A and second feature quantity calculating section303B. Region extracting section 304 includes first region extractingsection 304A and second region extracting section 304B. First featurequantity calculating section 303A calculates local structural featurequantities relating to the object image SO (step S104); second featurequantity calculating section 303B calculates local structural featurequantities relating to the reference image SR (step S105). A method forcalculating the structural feature quantities will be described later.

First feature quantity calculating section 303A and first regionextracting section 304A cooperate to extract partial object images PO1to PON (where N is an integer greater than or equal to 2) containinglocal structural features (for example moles, flecks, freckles, pores orpimples and pits that appear in facial skin) from the object image SOprovided from normalizing section 302 (step S106). Here, each of partialobject images PO1 to PON extracted may be a sub-region that is set basedon one point representing a local structural feature in the object imageSO. For example, a sub-region (for example, a circular or polygonalregion) centered at a point representing a local structural feature canbe extracted as a partial object image. Local coordinate positions,which will be described later, can be set in each of partial objectimages PO1 to PON.

On the other hand, second feature quantity calculating section 303B andsecond region extracting section 304B cooperate to extract partialreference images PR1 to PRM (where M is an integer greater than or equalto 2) containing local structural features (for example moles, flecks,freckles, pores or pimples and pits of the skin that appear in facialskin) from the reference image SR provided from normalizing section 302(step S107). Feature quantity calculating section 303 and regionextracting section 304 can constitute a feature image extracting sectionaccording to the present invention. Like partial object images PO1 toPON, each of partial reference images PR1 to PRM extracted may be asub-region that is set based on a point representing a local structuralfeature in reference image SR. For example, a sub-region (for example, acircular or polygonal region) centered at a point representing a localstructural feature can be extracted as a partial reference image. Localcoordinate positions, which will be described later, can be set in eachof partial reference images PR1 to PRM.

The number of partial object images PO1 to PON is not always greaterthan or equal to 2; it can be 0 or 1. Likewise, the number of partialreference images PR1 to PRM is not always greater than or equal to 2; itcan be 0 or 1. The process for extracting partial object images PO1 toPON from an object image SO and the process for extracting partialreference images PR1 to PRM from a reference image SR are collectivelyreferred to as the feature image extracting process.

The process from step S104 through step S107 will be described infurther detail. First feature quantity calculating section 303A setseach pixel in the object image SO as a pixel of interest P1 (p, q).First feature quantity calculating section 303A then determines a firstapproximate plane, which is a function z1 approximately representing aset of pixel values f1 (x, y) in a local region ΔS1 containing the pixelof interest P1 (p, q). Here, x and y are variables indicating thecoordinate position of a pixel value in local region ΔS1. First featurequantity calculating section 303A calculates a value proportional to thedifference Δ1 (p, q) between pixel value f1 (p, q) in object image SOand corresponding value z1 (p, q) in the first approximate plane (Δ1(p,q)=f1 (p, q)−z1 (p, q)) as a structural feature quantity g1 (p, q)relating to the object image SO (step S104). Structural featurequantities g1 (p, q) are calculated for all pixels in the object imageSO.

The array of structural feature quantities g1 (p, q) includes imageinformation in which a local structural feature is enhanced. Firstregion extracting section 304A can extract regions representing thelocal structural feature from the array of structural feature quantitiesg1 (p, q) as partial object images PO1 to PON (step S106).

Here, in order to compensate for shifts of pixel values due to externalfactors, it is desirable that the structural feature quantities g1 (p,q) be difference Δ1 (p, q) divided by statistical error s1 in thedifference (g1 (p, q)=Δ1/s1). The statistical error may be the standarddeviation, for example.

On the other hand, second feature quantity calculating section 303B setseach pixel in reference image SR as a pixel of interest P2 (p, q) anddetermines a second approximate plane which is a function z2approximately representing a set of pixel values f2 (x, y) in a localregion ΔS2 containing the pixel of interest P2 (p, q). Second featurequantity calculating section 303B calculates a value proportional to thedifference Δ2(p, q) between a pixel value f2 (p, q) in the referenceimage SR and corresponding value z2 (p, q) in the second approximateplane (Δ2(p, q)=f2 (p, q)−z2 (p, q)) as a structural feature quantity g2(p, q) relating to the reference image SR (step S105). Structuralfeature quantities g2 (p, q) are calculated for all pixels in thereference image SR.

The array of structural feature quantities g2 (p, q) includes imageinformation in which a local structural feature is enhanced. Secondregion extracting section 304B can extract regions representing thelocal structural feature from the array of structural feature quantitiesg2 (p, q) as partial reference images PR1 to PRM (step S107).

In order to compensate for shifts of pixel values due to externalfactors, it is desirable that the structural feature quantities g2 (p,q) be difference Δ2(p, q) divided by statistical error s2 in thedifference (g2 (p, q)=Δ2/s2). The statistical error may be the standarddeviation, for example.

The first and second approximate planes can be obtained by usingmultiple regression analysis. Here, let f (x, y) denote a pixel value f1(x, y) in an object image SO or a pixel value f2 (x, y) in a referenceimage SR. The function representing the first or second approximateplane is a linear function of variables x and y: z (x, y)=ax+by+c.Parameters a, b and c of the function can be determined as follows: thedifference between each function value z (x, y) and pixel value f (x, y)is squared and parameters a, b and c that result in the smallest sum ofthe squares for all x and y in local region ΔS1 or ΔS2 are obtained.

Structural feature quantity g (p, q) can be calculated according toEquation (1) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 1} \right\rbrack \mspace{599mu}} & \; \\{{g\left( {p,q} \right)} = \frac{{f\left( {p,q} \right)} - {z\left( {p,q} \right)}}{s}} & (1)\end{matrix}$

Here, structural feature quantity g (p, q) represents g1 (p, q) or g2(p, q) described above; s is the standard deviation of differences Δ1(x,y) in local region ΔS1 or the standard deviation of differences Δ2(x, y)in local region ΔS2.

A point representing a local structural feature can be a point with alocally low structural feature quantity. For example, for each pixel ofinterest in an image consisting of an array of structural featurequantities, the difference between the smallest structural featurequantity on the circumference of a circle centered at the pixel ofinterest with a certain radius and the structural feature quantity ofthe pixel of interest may be calculated. A pixel of interest thatsatisfies the condition in which the difference is greater than or equalto a threshold may be extracted as a feature point. With this, in facialimage matching, a mole, freckle or pore on skin texture, for example,can be extracted as a feature point.

Determination processing section 305 uses first image detecting section306 and second image detecting section 307 to perform determinationprocessing (step S108). Specifically, first image detecting section 306sets each of partial object images PO1 to PON as an image of interestand detects first partial image Ar that is most similar to the image ofinterest from a set Rg of partial reference images PR1 to PRM describedabove (the processing is referred to as “first image detectingprocessing”). Then, second image detecting section 307 detects secondpartial image Ao that is most similar to first partial image Ar from aset Og of partial object images PO1 to PON (the processing is referredto as “second image detecting processing”). Determination processingsection 305 determines whether or not the image of interest matchessecond partial image Ao and outputs the result of the determination toimage matching section 308 (the processing is referred to as“determination processing”). If determination processing section 305determines that the image of interest matches second partial image Ao,determination processing section 305 records the correspondencerelationship between first partial image Ar most similar to secondpartial image Ao and the image of interest in image correspondence table202 (the processing is referred to as “recording processing”).

The first image detecting processing, the second image detectingprocessing, the determination processing, and the recording processingare performed on all partial object images PO1 to PON.

FIG. 3 is a flowchart illustrating an example of a more specificprocedure of the determination processing. First, determinationprocessing section 305 skips step S201 and selects one unexaminedpartial image from set Og of partial object images PO1 to PON as animage of interest (step S202). Then, first image detecting section 306selects a group Rpg (subset) of partial reference images that are incoordinate positions close to the image of interest from set Rg ofpartial reference images PR1 to PRM (step S203). First image detectingsection 306 also detects first partial image Ar in partial referenceimage group Rpg that is most similar to the image of interest (stepS204).

Then, second image detecting section 307 selects group Opg of partialobject images that are in coordinate positions close to first partialimage Ar from set Og of partial object images PO1 to PON (step S205).Second image detecting section 307 then detects second partial image Aothat is most similar to first partial image Ar from partial object imagegroup Opg (step S206).

Determination processing section 305 determines whether or not the imageof interest matches second partial image Ao and outputs the result ofthe determination to image matching section 308 (step S207). Ifdetermination processing section 305 determines that the image ofinterest does not match second partial image Ao (NO at step S207),determination processing section 305 returns to step S201 and determineswhether all partial object images PO1 to PON have been subjected to thematching processing (step S201). If determination processing section 305determines that not all partial object images PO1 to PON have beensubjected to the matching processing (NO at step S201), determinationprocessing section 305 proceeds to step S202; if determinationprocessing section 305 determines that all partial object images PO1 toPON have been subjected to the matching processing (YES at step S201),determination processing section 305 ends the process.

On the other hand, if determination processing section 305 determinesthat the image of interest matches second partial image Ao (YES at stepS207), determination processing section 305 records the correspondencerelationship between first partial image Ar that is most similar tosecond partial image Ao and the image of interest in imagecorrespondence table 202 (step S208). Determination processing section305 then returns to step S201.

First image detecting section 306 can calculate a value representingstatistical correlation between the distribution of pixel values of animage of interest and the distribution of pixel values of each ofpartial reference images PR1 to PRM and can use the values as a measureof the similarity between the image of interest and each of partialreference images PR1 to PRM. Similarly, second image detecting section307 can use a value representing statistical correlation between thedistribution of pixel values of first partial image Ar and thedistribution of pixel values of each of partial object images PO1 to PONas a measure of the similarity between first partial image Ar and eachof partial object images PO1 to PON. The value representing statisticalcorrelation may be a correlation coefficient.

Let s (i, j) denote the measure of similarity between the i-th partialobject image POi and the j-th partial reference image PRj. At step S204(see FIG. 3) described above, partial reference image PRJ (where J isany number in the range of 1 to M) that is most similar to partialobject image POi that is an image of interest can be detected accordingto Equation (2) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 2} \right\rbrack \mspace{599mu}} & \; \\{J = {\arg \; {\max\limits_{j \in A_{1}}{s\left( {i,j} \right)}}}} & (2)\end{matrix}$

The equation provides the number j (=J) of partial reference image PRjthat results in the largest value of the measure of similarity s (i, j).Here, A1 is a set of the numbers j of the partial reference images thatbelong to partial reference image group Rpg.

If the measure of similarity s (i, j) is the correlation coefficient,the measure of similarity s (i, j) can be expressed by Equation (3)given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 3} \right\rbrack \mspace{599mu}} & \; \\{{s\left( {i,j} \right)} = \frac{\sum\limits_{a,b}{\left( {{g_{i}\left( {a,b} \right)} - {\langle g_{i}\rangle}} \right)\left( {{g_{j}\left( {a,b} \right)} - {\langle g_{j}\rangle}} \right)}}{\sqrt{\sum\limits_{a,b}{\left( {{g_{i}\left( {a,b} \right)} - {\langle g_{i}\rangle}} \right)^{2}{\sum\limits_{a,b}\left( {{g_{j}\left( {a,b} \right)} - {\langle g_{j}\rangle}} \right)^{2}}}}}} & (3)\end{matrix}$

Here a and b represent a local coordinate position set in the partialobject image or the partial reference image, gi (a, b) is the structuralfeature quantity in the local coordinate position (a, b) in partialobject image POi, gj (a, b) is the structural feature quantity in thelocal coordinate position (a, b) in partial reference image PRj, <gi> isthe average of the structural feature quantities gi (a, b) in thepartial object image POi, and <gj> is the average of the structuralfeature quantities gj (a, b) in partial reference image PRj.

Alternatively, first image detecting section 306 may calculate themeasure of similarity (i, j) as follows. First image detecting section306 calculates the noun (distance) ∥p1(m)−p2(n)∥ between a point p1(m)representing a local structural feature contained in the image ofinterest POi and a point p2(n) representing the local structural featurecontained in each partial reference image PRj. First image detectingsection 306 can then calculate the number of combinations (p1(m), p2(n))of points the calculated distance between which is less than or equal toa predetermined threshold as the measure of similarity s (i, j) betweenthe image of interest POi and each partial reference image PRj. Here,point p1(m) is a position vector (ai, bi) representing a localcoordinate position in image of interest POi and point p2(n) is aposition vector (aj, bj) representing a local coordinate position inpartial reference image PRj.

Here, the measure of similarity s (i, j) can be given by Equation (4)given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 4} \right\rbrack \mspace{599mu}} & \; \\{{s\left( {i,j} \right)} = {\sum\limits_{m \in B_{i}}{\sum\limits_{n \in B_{j}}{L\left( {m,n} \right)}}}} & (4)\end{matrix}$

where, Bi is a set of the numbers m of points p1(m) representing a localstructural feature contained in the image of interest POi and Bj is aset of the numbers n of points p2(n) representing the local structuralfeature contained in the partial reference image PRj.

First image detecting section 306 can calculate L (m, n) according toEquation (5) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 5} \right\rbrack \mspace{599mu}} & \; \\{{L\left( {m,n} \right)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} {{{p_{1}(m)} - {p_{2}(n)}}}} < {threshold}} \\0 & {otherwise}\end{matrix} \right.} & (5)\end{matrix}$

When the norm ∥p1(m)−p2(n)∥ between point p1(m) and point p2(n) is lessthan or equal to the threshold, the equation yields the value L(m, n) of“1”; otherwise, the equation yields the value of 0.

Like first image detecting section 306, second image detecting section307 can perform the following process. Second image detecting section307 calculates the distance between a point representing a localstructural feature contained in first partial image Ar and a pointrepresenting the local structural feature contained in each partialobject image. Second image detecting section 307 can use the number ofcombinations of points the calculated distance between which is lessthan or equal to a predetermined threshold as the measure of similaritybetween first partial image Ar and each partial object image.

The measure of similarity s (i, j) may be the similarity value obtainedaccording to Equation (3) multiplied by the similarity value obtainedaccording to Equation (4), for example.

At step 5206 (see FIG. 3) described above, second partial image Ao(=POK) that is most similar to first partial image Ar (=PRJ) can befound according to Equation (6) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 6} \right\rbrack \mspace{599mu}} & \; \\{K = {\arg \; {\max\limits_{k \in A_{2}}{s\left( {k,j} \right)}}}} & (6)\end{matrix}$

Here, A2 is a set of the numbers k of partial object images that belongto partial object image group Opg.

After completion of the determination processing (step S108 of FIG. 2)described above, image matching section 308 sums the values of themeasure of similarity s (i, j) between partial object images POi andpartial reference images PRj that have a correspondence relationshiprecorded in image correspondence table 202. Image matching section 308outputs the sum S as the degree of matching (matching score) (stepS109). The sum S can be calculated according to Equation (7) givenbelow.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 7} \right\rbrack \mspace{599mu}} & \; \\{S = {\sum\limits_{{({i,j})} \in C}{s\left( {i,j} \right)}}} & (7)\end{matrix}$

Here, set C consists of combinations (i, j) of partial object images POiand partial reference images PRj that have a correspondence relationshiprecorded in image correspondence table 202.

FIGS. 4 and 5 are diagrams illustrating feature spaces for explainingthe image matching described above. In the feature spaces depicted inthe Figs, circles represent feature quantities of images that belong toa set Rg of partial reference images; squares represent featurequantities of images that belong to a set Og of partial object images.As depicted in FIG. 4, if the i-th partial object image POi matches thej-th partial reference image PRj, the difference between partial objectimage POi and detected first partial image PRj is small. Consequently,second partial image POk that matches image of interest POi (that is,i=k) is likely to be detected at step S207 (see FIG. 3).

On the other hand, as depicted in FIG. 5, if a partial object image anda partial reference image do not match, the difference between partialobject image POi and detected first partial image PRj is large sincethere is no partial reference image that matches the image of interestPOi. Consequently, second partial image POk that does not match image ofinterest POi is likely to be detected.

As has been described above, image matching device 300 of the presentexemplary embodiment is capable of matching an object image against areference image with a high accuracy even if the position, shape orluminance of a local structural feature varies due to external factors.Since objects to be matched are in effect limited to partial objectimages PO1 to PON containing a local structural feature and partialreference images PR1 to PRM containing the local structural feature, thematching processing can be performed with a relatively small amount ofcomputation.

A variation of the exemplary embodiment described above will bedescribed below. FIG. 6 is a flowchart schematically illustrating aprocess procedure performed by image matching device 300 according tothe variation. The flowchart of FIG. 6 is the same as the flowchart ofFIG. 2, except that step S110 is provided between steps S108 and S109 inthe flowchart of FIG. 6.

At step S110, image matching section 308 assigns an appropriateweighting factor w (i, j) to the value of the measure of similarity s(i, j) between each partial object image and a partial reference imagethat have a correspondence relationship recorded in image correspondencetable 202. Image matching section 308 sums the weighted values of themeasure of similarity w (i, j) s (i, j) and outputs the sum S as thedegree of matching (matching score) (step S109).

Weighting factor w (i, j) is the number of combinations of points p1(m)representing a local structural feature contained in each partial objectimage and points p2(n) representing the local structural featurecontained in each partial reference image that satisfy the following twoconditions at the same time. One is that the distance between pointp1(m) and p2(n) is less than a predetermined threshold; the other isthat the combination of the points have a correspondence relationshiprecorded in image correspondence table 202. The weighting factor w (i,j) can be expressed by Equation (8) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 8} \right\rbrack \mspace{599mu}} & \; \\{{w\left( {i,j} \right)} = {\sum\limits_{m \in D_{i}}{\sum\limits_{n \in D_{j}}{L\left( {m,n} \right)}}}} & (8)\end{matrix}$

Here, L (m, n) can be calculated according to Equation (9) given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 9} \right\rbrack \mspace{599mu}} & \; \\{{L\left( {m,n} \right)} = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu} {{{p_{1}(m)} - {p_{2}(n)}}}} < {{threshold}\mspace{14mu} {and}\mspace{14mu} \left( {m,n} \right)}} \in C} \\0 & {otherwise}\end{matrix} \right.} & (9)\end{matrix}$

Therefore, matching score S can be calculated according to Equation (10)given below.

$\begin{matrix}{\left\lbrack {{expression}\mspace{14mu} 10} \right\rbrack \mspace{574mu}} & \; \\{S = {\sum\limits_{{({i,j})} \in C}{{w\left( {i,j} \right)}{s\left( {i,j} \right)}}}} & (10)\end{matrix}$

According to the variation, the feature points contained in set Di andset Dj in Equation (8) correspond to the combinations of partial imagesdetermined by determination processing section 305 to be in a stablecorrespondence with each other. Therefore, the influence ofunstably-extracted feature points and features points that are in anunstable correspondence with each other can be eliminated. Accordingly,matching with higher accuracy can be achieved.

While exemplary embodiments of the present invention have been describedwith reference to the drawings, the exemplary embodiments areillustrative of the present invention. Various other configurations maybe employed. For example, while first image extracting section 301A andsecond image extracting section 301B are separate functional blocks inthe exemplary embodiments, these functional blocks may be replaced witha single image extracting section that alternately generates a firstobject region image and a second object region image. Likewise, firstand second feature quantity calculating sections 303A and 303B and firstand second region extracting sections 304A and 304B may also be replacedwith such combined arrangements.

The image matching device of any of the exemplary embodiments describedabove can be used in applications such as an image search device thatsearches an image of a particular person from a group of images, inaddition to the application to a personal verification device usingbiometric information.

An example of the advantageous effects of the present invention will bedescribed below. In the image matching device, image matching method,and image matching program according to the present invention, one ormore partial object images containing a local structural feature areextracted from an object image and one or more partial reference imagescontaining the local structural feature are extracted from eachreference image. Each of the partial object images is set as an image ofinterest and a first partial image that is most similar to the image ofinterest is detected from the set of partial reference images and asecond partial image that is most similar to the first partial image isdetected from the set of partial object images. Since the image matchingdevice, the image matching method, and the image matching programdetermine whether or not the image of interest matches the secondpartial image, the result of the determination can be used to find apartial reference image that matches the partial object image withoutinconsistency. Therefore, matching an object image and a reference imagecan be accomplished with a high accuracy even if the position, shape orluminance of local structural feature varies due to external factors.

Since images to be matched are in effect limited to partial objectimages containing a local structural feature and partial referenceimages containing the local structural feature, the matching processingcan be performed with a relatively small amount of computation.

While the present invention has been described with respect to exemplaryembodiments thereof, the present invention is not limited to theexemplary embodiments described above. Various modifications which mayoccur to those skilled in the art can be made to the configurations anddetails of the present invention without departing from the scope of thepresent invention.

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2008-114395 filed on Apr. 24, 2008, thecontent of which is incorporated by reference.

1. An image matching device which matches an object image against one ormore reference images, comprising: a feature image extracting sectionextracting one or more partial object images containing a localstructural feature from the object image and extracting one or morepartial reference images containing a local structural feature from eachof the reference images; a first image detecting section setting each ofthe partial object images as an image of interest and detecting a firstpartial image most similar to the image of interest from a set of thepartial reference images; a second image detecting section detecting asecond partial image most similar to the first partial image from a setof the partial object images; and a determination processing sectiondetermining whether or not the image of interest matches the secondpartial image and outputting the result of the determination.
 2. Theimage matching device according to claim 1, wherein: the object imagecomprises a two-dimensional array of pixel values; the reference imagecomprises a two-dimensional array of pixel values; the feature imageextracting section comprises: a feature quantity calculating sectionsetting each pixel of the object image as a pixel of interest,determining a first approximate plane which is a function approximatelyrepresenting a set of pixel values in a local region containing thepixel of interest, and calculating a value proportional to a firstdifference between a pixel value in the local region and a value in thefirst approximate plane that corresponds to the pixel value as astructural feature quantity relating to the object image; and a regionextracting section extracting a region representing the local structuralfeature from an array of structural feature quantities relating to theobject image as the partial object image, wherein the feature quantitycalculating section sets each pixel of the reference image as a pixel ofinterest determines a second a approximate plane which is a functionapproximately representing set of pixel values in a local regioncontaining the pixel of interest, and calculates a value proportional toa second difference between a pixel value in the local region and avalue in the second approximate plane that corresponds to the pixelvalue as a structural feature quantity relating to the reference image;and the region extracting section extracts a region representing thelocal structural feature as the partial reference image from the arrayof the structural feature quantities relating to the reference image. 3.The image matching device according to claim 2, wherein: the featurequantity calculating section calculates a statistical error in the firstdifference relating to all pixel values in the local region for eachpixel of the object image and divides the first difference by thestatistical error in the first difference to obtain a structural featurequantity relating to the object image, and calculates a statisticalerror in the second difference relating to all pixel values in the localregion for each pixel of the reference image and divides the seconddifference by the statistical error in the second difference tocalculate a structural feature quantity relating to the reference image.4. (canceled)
 5. (canceled)
 6. (canceled)
 7. (canceled)
 8. The imagematching device according to claim 1, wherein: the first image detectingsection calculates a value representing a statistical correlationbetween the distribution of pixel values in the image of interest andthe distribution of pixel values in the partial reference image as ameasure of similarity between the image of interest and the partialreference image; and the second image detecting section uses the valuerepresenting the statistical correlation between the distribution ofpixel values in the first partial image and the distribution of pixelvalues in the partial object image as a measure of similarity betweenthe first partial image and the partial object image.
 9. (canceled) 10.(canceled)
 11. The image matching device according to claim 8, furthercomprising an image matching section calculating the degree of matchingbetween the object image and each of the reference images on the basisof the result of determination provided from the determinationprocessing section for all of the partial object images; wherein: thedetermination processing section records a correspondence relationshipbetween the image of interest and a first partial image most similar tothe second partial image in an image correspondence table when thedetermination processing section determines that the image of interestmatches the second partial image; and the image matching section sumsthe values of the measure of similarity between the partial objectimages and the partial reference images that have a correspondencerelationship recorded in the image correspondence table and outputs thesum as the degree of matching.
 12. The image matching device accordingto claim 1, wherein: the first image detecting section calculates thedistance between a first point representing the local structural featurecontained in the image of interest and a second point representing thelocal structural feature contained in each of the partial referenceimages and calculates the number of combinations of the first and secondpoints the calculated distance between which is less than or equal to apredetermined threshold as the measure of similarity between the imageof interest and each of the partial reference image; and the secondimage detecting section calculates the distance between a third pointrepresenting the local structural feature contained in the first partialimage and a fourth point representing the local structural featurecontained in each of the partial object images and uses the number ofcombinations of the third and fourth points the calculated distancebetween which is less than or equal to a predetermined threshold as themeasure of similarity between the first partial image and each of thepartial object images.
 13. (canceled)
 14. The image matching deviceaccording to claim 12, further comprising an image matching sectioncalculating the degree of matching between the object image and each ofthe reference images on the basis of the result of determinationprovided from the determination processing section for all of thepartial object images; wherein: the determination processing sectionrecords a correspondence relationship between the image of interest anda first partial image most similar to the second partial image in animage correspondence table when the determination processing sectiondetermines that the image of interest matches the second partial image;and the image matching section sums the values of the measure ofsimilarity between the partial object images and the partial referenceimages that have a correspondence relationship recorded in the imagecorrespondence table and outputs the sum as the degree of matching. 15.The image matching device according to claim 8, further comprising animage matching section calculating the degree of matching between theobject image and each of the reference images on the basis of the resultof determination provided from the determination processing section forall of the partial object images; wherein: the determination processingsection records a correspondence relationship between the image ofinterest and a first partial image most similar to the second partialimage in an image correspondence table when the determination processingsection determines that the image of interest matches the second partialimage; the image matching section assigns an appropriate weightingfactor to each value of the measure of similarity between the partialobject images and the partial reference images that have acorrespondence relationship recorded in the image correspondence table,sums the weighted values of the measure of similarity, and outputs thesum as the degree of matching; and the weighting factor satisfies acondition in which the distance between a first point representing thelocal structural feature contained in each of the partial object imagesand a second point representing the local structural feature containedin each of the partial reference images is less than or equal to apredetermined threshold and is the number of combinations of the firstand second points that have a correspondence relationship recorded inthe image correspondence table.
 16. (canceled)
 17. The image matchingdevice according to claim 1, wherein the local structural feature is atleast one feature selected from the group consisting of a mole, fleck,freckle, pore and skin irregularity that appears in facial skin.
 18. Theimage matching device according to claim 1, further comprising animage-to-be-matched extracting section extracting a first object regionimage from an input image and performing at least one operation fromamong: position adjustment, rotation and scaling of a subject image inthe first object region image on the basis of a global structuralfeature of the first object region image to generate the object image,wherein the image-to-be-matched extracting section extracts a secondobject region image from a registered image and performs at least oneoperation from among: position adjustment, rotation and scaling of asubject image in the second object region image on the basis of a globalstructural feature of the second object region image to generate thereference image.
 19. (canceled)
 20. The image matching device accordingto claim 18, wherein the global structural feature is a feature thatappears in facial skin.
 21. An image matching method for matching anobject image against one or more reference images, comprising:performing a feature image extracting step of extracting one or morepartial object images containing a local structural feature from theobject image and extracting one or more partial reference imagescontaining a local structural feature from each of the reference images;performing a first image detecting step of setting each of the partialobject images as an image of interest and detecting a first partialimage most similar to the image of interest from a set of the partialreference images; performing a second image detecting step of detectinga second partial image most similar to the first partial image from aset of the partial object images; and performing a determinationprocessing step of determining whether or not the image of interestmatches the second partial image and outputting the result of thedetermination.
 22. The image matching method according to claim 21,wherein: the object image comprises a two-dimensional array of pixelvalues; and the feature image extracting step comprises the steps of:setting each pixel of the object image as a pixel of interest,determining a first approximate plane which is a function forapproximately representing a set of pixel values in a local regioncontaining the pixel of interest, and calculating a value proportionalto a first difference between a pixel value in the local region and avalue in the first approximate plane that corresponds to the pixel valueas a structural feature quantity relating to the object image;extracting a region representing the local structural feature from anarray of structural feature quantities relating to the object image asthe partial object image; setting each pixel of the reference image as apixel of interest, determining a second approximate plane which is afunction for approximately representing a set of pixel values in a localregion containing the pixel of interest, and calculating a valueproportional to a second difference between a pixel value in the localregion and a value in the second approximate plane that corresponds tothe pixel value as a structural feature quantity relating to thereference image; and extracting a region representing the localstructural feature as the partial reference image from the array of thestructural feature quantities relating to the reference image.
 23. Theimage matching method according to claim 22, wherein: a structuralfeature quantity relating to the object image is calculated bycalculating a statistical error in the first difference relating to allpixel values in the local region for each pixel of the object image anddividing the first difference by the statistical error in the firstdifference; and a structural feature quantity relating to the referenceimage is calculated by calculating a statistical error in the seconddifference relating to all pixel values in the local region for eachpixel of the reference image and dividing the second difference by thestatistical error in the second difference.
 24. (canceled) 25.(canceled)
 26. (canceled)
 27. (canceled)
 28. The image matching methodaccording to claim 21, wherein: the first image detecting step comprisesthe step of calculating a value representing a statistical correlationbetween the distribution of pixel values in the image of interest andthe distribution of pixel values in the partial reference image as ameasure of similarity between the image of interest and the partialreference image; and the second image detecting step comprises the stepof using the value representing the statistical correlation between thedistribution of pixel values in the first partial image and thedistribution of pixel values in the partial object image as a measure ofsimilarity between the first partial image and the partial object image.29. (canceled)
 30. (canceled)
 31. The image matching method according toclaim 28, further comprising: when it is determined in the determinationprocessing step that the image of interest matches the second partialimage, recording a correspondence relationship between the image ofinterest and a first partial image most similar to the second partialimage in an image correspondence table; and when the determinationresults for all of the partial object images are given, summing thevalues of the measure of similarity between the partial object imagesand the partial reference images that have a correspondence relationshiprecorded in the image correspondence table and outputting the sum as thedegree of matching between the object image and each of the referenceimages.
 32. The image matching method according to claim 21, wherein:the first image detecting step comprises the steps of calculating thedistance between a first point representing the local structural featurecontained in the image of interest and a second point representing thelocal structural feature contained in each of the partial referenceimages and calculating the number of combinations of the first andsecond points the calculated distance between which is less than orequal to a predetermined threshold as the measure of similarity betweenthe image of interest and each of the partial reference images; and thesecond image detecting step comprises the steps of calculating thedistance between a third point representing the local structural featurecontained in the first partial image and a fourth point representing thelocal structural feature contained in each of the partial object imagesand using the number of combinations of the third and fourth points thecalculated distance between which is less than or equal to apredetermined threshold as the measure of similarity between the firstpartial image and each of the partial object images.
 33. (canceled) 34.The image matching method according to claim 32, further comprising:when it is determined in the determination processing step that theimage of interest matches the second partial image, recording acorrespondence relationship between the image of interest and a firstpartial image most similar to the second partial image in an imagecorrespondence table; and when the determination results for all of thepartial object images are given, summing the values of the measure ofsimilarity between the partial object images and the partial referenceimages that have a correspondence relationship recorded in the imagecorrespondence table and outputting the sum as the degree of matching.35. The image matching method according to claim 28, further comprising:when it is determined in the determination processing step that theimage of interest matches the second partial image, recording acorrespondence relationship between the image of interest and a firstpartial image most similar to the second partial image in the imagecorrespondence table; and when the determination results for all of thepartial object images are given, assigning an appropriate weightingfactor to each value of the measure of similarity between the partialobject images and the partial reference images that have acorrespondence relationship recorded in the image correspondence table,summing the weighted values of the measure of similarity, and outputtingthe sum as the degree of matching; wherein the weighting factorsatisfies a condition in which the distance between a first pointrepresenting the local structural feature contained in each of thepartial object images and a second point representing the localstructural feature contained in each of the partial reference images isless than or equal to a predetermined threshold and is the number ofcombinations of the first and second points that have a correspondencerelationship recorded in the image correspondence table.
 36. (canceled)37. The image matching method according to claim 21, wherein the localstructural feature is at least one feature selected from the groupconsisting of a mole, fleck, freckle, pore and skin irregularity thatappears in facial skin.
 38. The image matching method according to claim21, further comprising: extracting a first object region image from aninput image and performing at least one operation from among: positionadjustment, rotation and scaling of a subject image in the first objectregion image on the basis of a global structural feature of the firstobject region image to generate the object image; and extracting asecond object region image from a registered image and performing atleast one operation from among: position adjustment, rotation andscaling of a subject image in the second object region image on thebasis of a global structural feature of the second object region imageto generate the reference image.
 39. (canceled)
 40. The image matchingmethod according to claim 38, wherein the global structural feature is afeature that appears in facial skin.
 41. An image matching programproduct causing a computer to execute a process for matching an objectimage against one or more reference images, the program comprising: afeature image extracting step of extracting one or more partial objectimages containing a local structural feature from the object image andextracting one or more partial reference images containing a localstructural feature from each of the reference images; a first imagedetecting step of setting each of the partial object images as an imageof interest and detecting a first partial image most similar to theimage of interest from a set of the partial reference images; a secondimage detecting step of detecting a second partial image most similar tothe first partial image from a set of the partial object images; and adetermination processing step of determining whether or not the image ofinterest matches the second partial image and outputting the result ofthe determination.
 42. The image matching program product according toclaim 41, wherein: the object image comprises a two-dimensional array ofpixel values and the reference image comprises a two-dimensional arrayof pixel values; and the feature image extracting step comprises: afirst feature quantity calculating step of setting each pixel of theobject image as a pixel of interest, determining a first approximateplane which is a function for approximately representing a set of pixelvalues in a local region containing the pixel of interest, andcalculating a value proportional to a first difference between a pixelvalue in the local region and a value in the first approximate planethat corresponds to the pixel value as a structural feature quantityrelating to the object image; a second region extracting step ofextracting a region representing the local structural feature from anarray of structural feature quantities relating to the object image asthe partial object image; a second feature quantity calculating step ofsetting each pixel of the reference image as a pixel of interest,determining a second approximate plane which is a function forapproximately representing a set of pixel values in a local regioncontaining the pixel of interest, and calculating a value proportionalto a second difference between a pixel value in the reference image anda value in the second approximate plane that corresponds to the pixelvalue as a structural feature quantity relating to the reference image;and a second region extracting step of extracting a region representingthe local structural feature as the partial reference image from thearray of the structural feature quantities relating to the referenceimage.
 43. The image matching program product according to claim 42,wherein: a structural feature quantity relating to the object image iscalculated by calculating a statistical error in the first differencerelating to all pixel values in the local region for each pixel of theobject image and dividing the first difference by the statistical errorin the first difference; and a structural feature quantity relating tothe reference image is calculated by calculating a statistical error inthe second difference relating to all pixel values in the local regionfor each pixel of the reference image and dividing the second differenceby the statistical error in the second difference.
 44. (canceled) 45.(canceled)
 46. (canceled)
 47. (canceled)
 48. The image matching programproduct according to claim 41, wherein: the first image detecting stepcomprises the step of calculating a value representing a statisticalcorrelation between the distribution of pixel values in the image ofinterest and the distribution of pixel values in the partial referenceimage as a measure of similarity between the image of interest and thepartial reference image; and the second image detecting step comprisesthe step of using the value representing the statistical correlationbetween the distribution of pixel values in the first partial image andthe distribution of pixel values in the partial object image as ameasure of similarity between the first partial image and the partialobject image.
 49. (canceled)
 50. (canceled)
 51. The image matchingprogram product according to claim 48, further comprising the steps of:when it is determined by the determination processing step that theimage of interest matches the second partial image, recording acorrespondence relationship between the image of interest and a firstpartial image most similar to the second partial image in an imagecorrespondence table; and when the determination results for all of thepartial object images are given, summing the values of the measure ofsimilarity between the partial object images and the partial referenceimages that have a correspondence relationship recorded in the imagecorrespondence table and outputting the sum as the degree of matchingbetween the object image and each of the reference images.
 52. The imagematching program product according to claim 41, wherein: the first imagedetecting step comprises the steps of calculating the distance between afirst point representing the local structural feature contained in theimage of interest and a second point representing the local structuralfeature contained in each of the partial reference images andcalculating the number of combinations of the first and second pointsthe calculated distance between which is less than or equal to apredetermined threshold as the measure of similarity between the imageof interest and each of the partial reference image; and the secondimage detecting step comprises the steps of calculating the distancebetween a third point representing the local structural featurecontained in the first partial image and a fourth point representing thelocal structural feature contained in each of the partial object imagesand using the number of combinations of the third and fourth points thecalculated distance between which is less than or equal to apredetermined threshold as the measure of similarity between the firstpartial image and each of the partial object images.
 53. (canceled) 54.The image matching program product according to claim 52, furthercomprising the steps of: when it is determined by the determinationprocessing step that the image of interest matches the second partialimage, recording a correspondence relationship between the image ofinterest and a first partial image most similar to the second partialimage in an image correspondence table; and when the determinationresults for all of the partial object images are given, summing thevalues of the measure of similarity between the partial object imagesand the partial reference images that have a correspondence relationshiprecorded in the image correspondence table and outputting the sum as thedegree of matching.
 55. The image matching program product according toclaim 48, further comprising the steps of: when it is determined in thedetermination processing step that the image of interest matches thesecond partial image, recording a correspondence relationship betweenthe image of interest and a first partial image most similar to thesecond partial image in the image correspondence table; and when thedetermination results for all of the partial object images are given,assigning an appropriate weighting factor to each value of the measureof similarity between the partial object images and the partialreference images that have a correspondence relationship recorded in theimage correspondence table, summing the weighted values of the measureof similarity, and outputting the sum as the degree of matching; whereinthe weighting factor satisfies a condition in which the distance betweena first point representing the local structural feature contained ineach of the partial object images and a second point representing thelocal structural feature contained in each of the partial referenceimages is less than or equal to a predetermined threshold and is thenumber of combinations of the first and second points that have acorrespondence relationship recorded in the image correspondence table.56. (canceled)
 57. The image matching program product according to claim41, wherein the local structural feature is at least one featureselected from the group consisting of a mole, fleck, freckle, pore andskin irregularity that appears in facial skin.
 58. The image matchingprogram product according to claim 41, further comprising: a first imageextracting step of extracting a first object region image from an inputimage and performing at least one operation from among: positionadjustment, rotation and scaling of a subject image in the first objectregion image on the basis of a global structural feature of the firstobject region image to generate the object image; and a second imageextracting step of extracting a second object region image from aregistered image and performing at least one operation from among:position adjustment, rotation and scaling of a subject image in thesecond object region image on the basis of a global structural featureof the second object region image to generate the reference image. 59.(canceled)
 60. The image matching program product according to claim 58,wherein the global structural feature is a feature that appears infacial skin.